1. Field of the Invention
The invention relates to motion vector detecting apparatus and method which can detect a motion vector of a below-pixel precision by executing a prediction of a pixel value at a position that is more detailed than a pixel position from an input image signal, a prediction of a motion evaluation value at a below-pixel position from a motion evaluation value at the pixel position, or the like by using a class sort adaptive process.
2. Description of the Related Art
Recently, an importance degree of a motion vector detection for a digital image as a target is more and more increasing. This is because it greatly contributes to a practical use. For example, a motion compensation is generally performed in a recent image compression. The improvement of a precision of the motion vector which is used for the motion compensation results in an improvement in a compression efficiency. Therefore, various methods have been proposed with respect to a motion vector detecting method. Generally, the motion vector detecting method for a motion image as a target is mainly classified into the following three kinds of methods.
The first motion vector detecting method is a block matching method. In the block matching method, a present image and a past image are compared in order to discriminate whether a region of the present image which was divided into blocks existed in the past image by the same idea as a pattern matching. As a specific example, a differential absolute value of every corresponding pixel in the block is added and a position where a differential absolute value sum of every block is minimum is set to the motion vector. According to the above method, although a detecting precision is high, there is a drawback such that an operation amount is extremely large.
The second motion vector detecting method is a gradient method. The gradient method is based on a model such that when a pixel having a certain space inclination moves to a certain position, a time difference according to a motion amount occurs. Therefore, by dividing the time difference by the space inclination, a motion vector is obtained. This method is excellent in a point such that an operation amount is small. However, there is a drawback such that when the motion amount increases, a precision deteriorates. This is because the above model is not satisfied.
The third motion vector detecting method is a phase correlation method. The phase correlation method is a method whereby a Fourier transformation is performed to block data at the same position of the present image and the past image, a deviation amount of the phase is detected in a frequency region, and an inverse Fourier transformation is subsequently executed, thereby detecting the motion vector from information of a phase term thereof. As characteristics of the method, in order to assure a precision, a large block size of a certain extent or more is required. Therefore, an operation amount by the Fourier transformation is remarkably large. Moreover, there is a drawback such that a possibility in which a plurality of motion objects exist in a large block is generally high, so that the discrimination is difficult. Since the precision of the motion vector is a target pixel precision of the Fourier transformation, only the motion vector of an input pixel pitch can be obtained.
A constructional example in which a motion vector of a pixel precision is detected by using the phase correlation method will now be described with reference to FIG. 1. An image signal d11 is supplied from an input terminal 11. The image signal d11 is supplied to a block dividing unit 12. The supplied image signal d11 is divided into predetermined blocks by the block dividing unit 12. An image signal d12 which was divided into blocks is supplied from the block dividing unit 12 to a Fourier transforming unit 13 and a frame memory 15. In the Fourier transforming unit 13, a Fourier transformation is executed to the supplied image signal d12. In the frame memory 15, the supplied image signal d12 is delayed by one frame and, after that, is supplied to a Fourier transforming unit 16 as an image signal d16.
In the Fourier transforming unit 16, a Fourier transformation is executed to the one-frame preceding block data processed by the Fourier transforming unit 13. Operation results d13 and d17 of the Fourier transforming units 13 and 16 are supplied to a phase correlation detecting unit 14. The phase correlation detecting unit 14 forms a phase correlation (phase difference matrix) exp(-j.phi.) which will be described hereinlater in accordance with the two operation results d13 and d17 which are supplied. The formed phase correlation exp(-j.phi.) is supplied to a vector detecting unit 17. In the vector detecting unit 17, an inverse Fourier transformation and a peak detection of a phase correlation function are executed to the phase correlation exp(-j.phi.) and a motion vector d15 can finally be obtained. The motion vector d15 is outputted from an output terminal 18.
However, a precision of the motion vector which is detected by using the phase correlation is set to one pixel unit. In case of the motion compensation, for example, in order to execute a compensation at a high precision, it is necessary to detect a motion vector of a precision higher than one pixel.
A prior art of a motion vector detecting apparatus of a below-pixel precision will now be shown.
According to the above motion vector detecting apparatus, a motion vector of the pixel precision is detected by using the block matching method and, on the basis of a motion evaluation value which is formed at that time, a motion vector of a below-pixel precision is detected. First, for an explanation of the block matching method, a constructional example of the block data is shown in FIG. 2. A case where a motion vector of a certain block is detected in an adjacent frame is considered. In a #k frame and a #(k-1) frame, a block of a size of (M pixels.times.N lines) is set to a spatially corresponding position.
A pattern matching is performed between the block of the #k frame and the block at each coordinate of the #(k-1) frame, thereby detecting optimum coordinates. A differential absolute value sum or the like of the corresponding pixel in the block of a size of (M pixels.times.N lines) corresponding to each position is used for evaluation. Assuming that each pixel level of the #k frame is set to L.sub.n (i, j) and each pixel level of the #(k-1) frame is set to L.sub.n-l (i, j), the following equation (1) or the like can be mentioned as an example of an evaluating equation at coordinates (x, y).
Evaluating equation: ##EQU1##
In an example of FIG. 2, a value of a motion evaluation value E of the equation (1) is calculated at each coordinate of an X.multidot.Y point. In the coordinates of the X.multidot.Y point, coordinates in which the evaluation value indicates the minimum value become a motion vector. Since the evaluation value which is calculated here relates to each pixel, a motion vector which is detected is based on a pixel precision. Subsequently, an example in which a motion vector of a below-pixel precision is detected on the basis of a motion evaluation value at a position near the coordinates of the detected motion vector of the pixel precision is shown.
For explanation, an example in which the motion evaluation value of the equation (1) is detected at a 1-dimensional coordinate position is shown in FIG. 3. It is assumed that motion evaluation values of the equation (1) at coordinates X0, X1, and X2 are set to E0, E1, and E2. Since the motion vector of the pixel precision corresponds to coordinates in which the motion evaluation value is minimum, in this case, it is set to X0. As an example in which the motion vector of the below-pixel precision is detected by using the motion evaluation values E0, E1, and E2, it is considered that a fluctuation value .DELTA.X below the pixel is obtained by a linear interpolation on the basis of E0, E1, and E2 as shown in the diagram. A calculating equation in this case is expressed by,the following equation (2).
Fluctuation value: EQU .DELTA.X=(E2-E1)/{2.times.(E2-E0)} (2) PA1 memory means for storing predictive coefficients which have previously been obtained by learning; and PA1 predictive value forming means for forming an optimum predictive value from an arithmetic operation of a predictive equation using the predictive coefficient read out in correspondence to a class and an input pixel value or a motion evaluation value, PA1 and a center of gravity method comprising:
Thus, the motion vector of the below-pixel precision in FIG. 3 is set to (X0-.DELTA..times.). A constructional example of the above process is shown in FIG. 4. Pixel data d21 is supplied from an input terminal 21. The pixel data d21 is supplied to a memory unit 22 and a detecting unit 23. In the memory unit 22, the pixel data d21 is delayed for a predetermined period of time and is supplied as delayed pixel data d22 to the detecting unit 23. In the detecting unit 23, a motion vector of a pixel precision is detected from the pixel data d21 and d22. In case of using the block matching method, the calculation of the motion evaluation value of the equation (1) and the detection of the minimum value are executed and the motion vector of the pixel precision is obtained.
Subsequently, in a detecting unit 24, on the basis of the result obtained from the detecting unit 23, namely, by using the motion vector of the pixel precision as a reference, a motion vector of a below-pixel precision is detected by using the motion evaluation value of the pixel precision. In this case, an arithmetic operation of the equation (2) is executed. The motion vector of the below-pixel precision is finally obtained from an output terminal 25 in this manner.
However, there is actually a problem such that a motion vector having an enough precision cannot be obtained by the linear interpolation mentioned above.